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Executive  Summary 


This  report  describes  the  development  and  user  evaluation  of  pictorial  symbols  representing  32 
Federal  Aviation  Administration  (FAA)  facilities  and  services.  The  potential  users,  Airway 
Facilities  (AF)  system  specialists,  evaluated  these  symbols  objectively  and  subjectively.  After 
completing  the  evaluation  and  analysis  process,  the  researchers  created  a  recommended  set  of 
symbols  for  AF  use. 

The  development  and  evaluation  process  consisted  of  a  literature  survey,  a  brainstorming  session 
involving  subject  matter  experts  (SMEs),  creation  of  software,  and  a  field  study.  The  literature 
survey  provided  a  basis  for  the  development  of  symbols  and  the  preparation  of  evaluation 
procedures.  The  researchers  designed  symbols  to  represent  32  facilities  and  services  using 
information  gained  from  the  literature  survey  and  the  advice  of  SMEs. 

Eleven  specialists  participated  in  the  symbol  evaluation  at  the  FAA  William  J.  Hughes  Technical 
Center.  Seventeen  AF  specialists  and  managers  participated  in  the  symbol  evaluation  at  Dallas- 
Fort  Worth  Air  Route  Traffic  Control  Center  (ARTCC)  and  Prototype  Operational  Control 
Center.  The  groups  were  similar  in  age,  experience,  and  education. 

The  participants  were  trained  on  recognition  and  recall  of  the  symbols.  Following  the  training 
sessions,  the  participants  rated  each  symbol’s  representation  of  the  facility  or  service,  learning 
difficulty,  and  distinctiveness.  The  researchers  then  conducted  recognition  and  recall  testing, 
recording  the  number  of  errors  the  participants  made  for  each  symbol.  Finally,  the  researchers 
collected  exit  preference  data  using  an  Exit  Questionnaire,  which  let  the  participants  choose  a 
preferred  symbol  from  the  symbols  developed  for  each  facility  or  service. 

The  research  team  used  three  criteria  to  determine  which  of  the  two  symbols  developed  for  each 
facility  and  service  is  better.  The  first  selection  criterion  was  the  number  of  recognition  errors. 
The  second  selection  criterion  was  the  participant  ratings  of  the  symbols.  The  exit  preference 
data  provided  the  third  selection  criterion  to  differentiate  the  better  symbol  if  the  two  other 
criteria  had  not  made  clear  which  was  the  better  symbol. 

After  evaluating  the  symbols  using  the  three  selection  criteria,  the  researchers  developed  a  set  of 
recommended  visual  symbols  for  use  in  AF  displays.  Use  of  these  symbols  will  help  standardize 
display  interfaces  across  the  AF  environment. 
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1.  Introduction 


The  Federal  Aviation  Administration  (FAA)  plans  to  increase  the  number  of  Airway  Facility 
(AF)  systems  remotely  monitored  and  controlled  at  centralized  locations.  Therefore,  AF  system 
specialists  at  these  locations  must  recognize  and  respond  to  information  about  an  increasing 
number  of  services  and  facilities.  This  creates  a  problem  about  how  to  represent  the  information. 
Personal  computer  software  routinely  uses  visual  symbols  to  identify  software  applications  and 
computer  functions.  The  FAA  has  started  to  use  a  similar  approach  to  represent  information 
about  facilities  and  services  on  computer  displays.  However,  standardized  symbols  representing 
facilities  and  services  do  not  currently  exist.  Lack  of  standardization  may  cause  AF  specialists  to 
encounter  different  symbols  representing  the  same  object  or  the  same  symbol  representing 
different  objects.  Furthermore,  it  may  increase  time  to  learn  the  symbols  and  increase  the 
possibility  of  error  in  interpreting  the  symbols.  The  adoption  of  standardized  symbols  that  have 
been  evaluated  for  usability  would  alleviate  these  problems,  thereby  improving  human 
performance. 

1 . 1  Background 

The  development  and  assessment  of  AF  symbols  began  in  1994.  Initially  FAA  psychologists,  AF 
consultants,  and  human  factors  personnel  developed  symbols  for  AF.  Phase  I  of  this  work 
represented  a  first  step  in  an  effort  to  develop  a  set  of  standard  symbols  for  use  throughout  AF, 
resulting  in  a  literature  review  of  the  human  factors  literature  on  visual  and  auditory  symbols 
(Duncanson,  1994).  Phase  n  evaluated  a  set  of  121  symbols  resulting  in  a  recommended  symbol 
set  of  38  pictorial  symbols  for  AF  use  (Duncanson  et  al.,  1996).  This  report  is  a  continuation  of 
the  work  presented  in  the  initial  two  reports,  looking  at  32  symbols  representing  facilities  and 
services  not  represented  in  the  Phase  n  study. 

1 .2  Purpose  and  Rationale 

The  project  purpose  was  to  develop,  evaluate,  and  recommend  symbols  based  on  their  suitability 
to  support  computer-human  interface  requirements  within  the  AF  domain. 

2.  Method 

This  section  explains  the  development  process,  evaluation  techniques,  and  the  formulation  of 
evaluation  criteria  for  this  project. 

2. 1  Participants 

The  participants  included  system  specialists  and  managers  from  several  FAA  regions.  They 
included  1 1  AF  system  specialists  who  visited  the  FAA  Research  Development  and  Human 
Factors  Laboratory  at  the  FAA  William  J.  Hughes  Technical  Center.  Seventeen  AF  specialists 
and  managers  from  the  Dallas  Fort  Worth  Air  Route  Traffic  Control  Center  (ARTCC)  and 
Prototype  Operational  Control  Center  (POCC)  participated  in  the  symbol  evaluation  at  their 
location.  The  researchers  divided  the  28  participants  into  two  groups  of  14.  Each  group 
evaluated  one  of  two  sets  of  symbols  (referred  to  as  Group  A  and  Group  B)  for  facilities  and 
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services.  The  groups  were  similar  in  experience,  age,  and  education.  The  researchers  briefed  all 
participants  on  the  purpose  of  the  study  and  the  confidentiality  of  the  test  data.  All  participants 
gave  an  oral  acceptance  for  participating  in  the  study. 

2.2  Apparatus 

The  research  team  developed  a  software  tool  to  support  data  collection,  training,  and  testing  for 
the  symbol  research.  The  software  presented  grayscale  symbols  that  were  .5  in.  x  .5  in.  square 
(51  pixels  X  51  pixels).  The  symbols  were  presented  on  a  Toshiba  TECRA  720CDT  laptop  with  a 
.28  mm.  dot  pitch  and  a  resolution  of  1028  x  1028. 

2.3  Procedure 

2.3.1  Symbol  Development 

Former  FAA  AF  employees  served  as  subject  matter  experts  (SMEs)  and  assisted  the  research 
team  in  identifying  symbols  requiring  development.  The  team  reviewed  previously  developed 
symbols  and  identified  AF  facilities  and  services  requiring  new  or  enhanced  symbols.  Because 
some  facilities  have  similar  functions,  the  SMEs  recommended  developing  one  symbol  to 
represent  groups  of  facilities  (e.g.,  combining  visual  aid  facilities  into  a  common  symbol). 
Appendix  A  describes  these  combined  symbols. 

The  research  team  reviewed  industry  and  government  documents  to  provide  a  basis  for  symbol 
development  and  evaluation  activities.  The  literature  search  revealed  that  brainstorming  sessions 
are  used  frequently  to  develop  symbols  for  applications  (Crist  &  Aurelio,  1990;  Howard, 
O’Boyle,  Eastman,  Andre,  &  Motoyama,  1991;  Microsoft,  1995).  The  research  team  decided  to 
adopt  the  same  approach  for  the  development  of  symbols  for  this  study. 

SME  expertise,  literature  survey  insights,  and  the  Guidelines  for  Developing  Symbols  in  Airway 
Facilities  by  Ramakrishnan,  Cranston,  and  Grayson  (1997)  contributed  to  the  brainstorming 
sessions.  The  researchers  derived  symbols  through  the  adoption  or  modification  of  existing 
symbols  or  through  the  brainstorming  sessions.  They  designed  and  modified  symbols  so  that  the 
symbols  either  had  a  physical  resemblance  to  or  an  association  with  the  actual  facility  or  service 
they  represented  or  depicted  a  functional  aspect  of  the  facility  or  service  represented.  The 
developers  standardized  the  representation  of  clouds,  aircraft,  control  towers,  ARTCC  buildings, 
and  borders  used  within  the  symbols.  The  researchers  developed  two  symbols  for  each  facility  or 
service  except  the  airport  and  regional  facilities,  for  which  only  one  symbol  each  was  developed. 
Pilot  studies  had  indicated  that  the  symbols  for  Airport  and  Region  were  acceptable  and  thus 
submitted  each  one  to  testing  without  an  alternative. 

2.3.2  Training 

The  participants  learned  the  symbols  using  first  recognition  and  then  recall  training.  Recognition 
training  began  with  a  briefing  on  the  features  of  the  symbol.  Using  a  task  similar  to  a  multiple 
choice  test,  a  participant  viewed  a  facility  or  service  name  on  the  screen  with  five  randomly 
positioned  symbols  (one  target  symbol  and  four  distracter  symbols;  see  Figure  1).  The 
participant’s  task  was  to  click  on  the  symbol  representing  the  named  facility  or  service.  The 
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participant  continued  to  click  on  the  symbols  until  he/she  picked  the  correct  symbol.  Once  a 
participant  picked  the  correct  symbol,  the  facility/service  name  and  the  correct  symbol  remained 
on  the  screen  for  3  seconds  to  reinforce  the  learning  process.  The  researchers  considered  this 
symbol  identification  process  to  be  one  cycle  of  testing  for  analysis  purposes.  The  task  of 
recognizing  the  correct  symbol  continued  until  the  participant  identified  each  symbol  correctly  on 
two  consecutive  cycles. 


Figure  1.  Computer  display  used  for  recognition  training. 

The  participants  then  took  part  in  recall  training  (see  Figure  2).  Each  participant  orally  recalled 
the  name  of  the  facility  or  service  when  presented  with  its  respective  symbol.  The  researchers 
provided  the  participant  with  the  correct  name  for  the  facility  or  service  after  each  response  and 
recorded  any  errors  made  in  remembering  the  facility  or  service  name.  Presentation  order  of  the 
symbols  was  randomized  to  prevent  order  effects.  The  symbol  presentation  continued  until  the 
participant  correctly  recalled  the  complete  name  of  the  facility  or  service  represented  by  the 
symbol  on  two  consecutive  cycles.  The  researchers  considered  training  complete  when  the 
participants  successfully  completed  the  recognition  and  recall  tasks  for  all  the  symbols. 


Figure  2.  Computer  display  used  to  collect  recall  data. 
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2.3.3  Symbol  Evaluation 


Researchers  have  used  both  objective  and  subjective  methods  to  assess  symbol  quality. 

Kaufmann  and  Eaton  (1994)  employed  user  preference  to  study  symbols  for  a  marine  navigation 
electronic  chart  display  and  information  system.  Researchers  such  as  Whitaker  (1985)  and 
Kirkpatrick,  Dutra,  Heasly,  Granda,  and  Vingelis  (1992)  used  accuracy  and  speed  as  objective 
indicators  of  symbol  goodness.  Blackwell  and  Cuomo  (1991)  used  search  time  and  errors  as 
variables  on  a  space  and  missile  warning  symbol  set.  Green  and  Pew  (1978),  Kantowitz  and 
Sorkin  (1983),  and  Aurelio  and  Crist  (1990)  constructed  confusion  matrices  based  on  error  data. 
Other  symbol  evaluation  techniques  have  included  using  focus  groups  and  usability  assessments. 

The  research  team  for  this  study  decided  to  use  a  combination  of  recall  and  recognition  testing, 
subjective  ratings,  and  user  preference  to  evaluate  the  quality  of  the  symbols.  They  began  the 
study  by  collecting  biographical  information  about  the  participants  and  data  related  to  participant 
familiarity  with  each  facility  and  service.  The  researchers  presented  each  participant  with  a  brief 
introduction  to  all  the  symbols.  They  discussed  how  the  symbol  represented  the  actual  facility  or 
service,  functions  depicted,  and  associative  relationships  of  the  symbol  components.  The 
participants  were  asked  to  rate  their  familiarity  with  the  associated  facility  or  service  on  a  scale  of 
1  (not  familiar)  to  7  (high  familiarity).  Figure  3  presents  the  computer  display  used  to  collect 
familiarity  data. 


Figure  3.  Computer  display  used  to  collect  familiarity  data. 

The  participants  were  then  trained  with  the  different  symbols  as  described  in  Section  2.3.2. 
Following  the  training  sessions,  the  researchers  collected  ratings  on  how  well  each  symbol 
represented  a  facility  or  service,  its  learning  difficulty,  and  how  easily  it  could  be  confused  with 
other  symbols.  The  rating  scales  ranged  from  one  to  seven  with  a  rating  of  seven  indicating  the 
symbol  was  an  excellent  representation,  very  easy  to  learn,  and  not  confusing  with  other  symbols 
(see  Appendix  B  for  an  example). 
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After  the  ratings  data  collection,  participants  took  a  10-minute  break  to  minimize  some  of  the 
short  memory  storage  that  would  have  occurred  if  testing  had  immediately  followed.  The  symbol 
testing  followed  the  break. 

The  researchers  used  a  multiple  choice  recognition  task  to  test  the  ability  of  the  participants  to 
recognize  each  symbol.  The  test  process  was  similar  to  the  recognition  training  except  there 
were  16  multiple  choices  (see  Figure  4).  The  researchers  used  16  symbol  choices  because  the  AF 
SMEs  thought  that  a  typical  operational  display  reflecting  degraded  or  out-of-service  facilities 
might  contain  approximately  16  different  symbols.  The  software  immediately  moved  to  the  next 
test  symbol  after  selection  of  a  correct  symbol.  Testing  continued  until  participants  correctly 
identified  each  symbol  twice  consecutively. 


Figure  4.  Computer  display  of  recognition  testing. 

Before  recall  testing,  researchers  instructed  that  accuracy  was  the  most  important  consideration 
when  recalling  symbol  names.  Recall  testing  repeated  the  training  recall  step  with  trials 
continuing  until  the  participants  recalled  the  full  name  of  each  symbol  correctly.  The  researcher 
recorded  a  participant’s  response  even  if  erroneous. 

The  paper  and  pencil  Exit  Questionnaire  provided  participants  with  an  opportunity  to  select 
between  two  symbols,  express  opinions,  and  identify  preferred  symbol  characteristics  (see 
Appendix  C).  The  participants  were  given  a  randomized  sheet  of  Group  A  and  Group  B 
symbols.  The  symbol  selection  exercise  asked  participants  to  indicate  a  preference  for  either  the 
symbol  their  group  had  learned  or  the  symbol  that  had  been  learned  by  the  other  group. 
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3.  Results 


3.1  Subjective  rating 

Table  1  shows  that  Representation  (how  well  the  symbol  represented  the  service/facility), 
Difficulty  (how  difficult  the  symbol  was  to  learn),  and  Distinctiveness  (how  easily  the  target 
symbol  was  confused  with  other  symbols)  were  all  highly  correlated.  Therefore,  the  researchers 
averaged  the  three  ratings  to  create  a  composite  rating  for  each  symbol. 


Table  1 .  Regression  Analysis  Results 


Representation 

Difficulty 

Distinctiveness 

Representation 

1.000 

- 

- 

Difficulty  To  Learn 

0.836 

1.000 

- 

Distinctiveness 

0.845 

0.826 

1.000 

3.2  Symbol  Selection 

Table  2  shows  the  symbol  selection  based  on  the  three  criteria  of  errors,  average  rating,  and  exit 
preference.  The  number  of  recognition  errors  was  the  first  criterion  used  for  selecting  between 
two  symbols'.  When  recognition  errors  did  not  clearly  depict  a  better  symbol,  the  researchers 
used  a  second  criterion  based  on  the  average  rating  of  subjective  symbol  qualities.  The 
researchers  set  an  average  rating  of  3  or  less  as  the  failure  criterion  for  a  symbol.  Based  on  this 
criterion,  symbols  with  an  average  rating  of  3  or  less  are  determined  to  be  too  poor 
representations,  too  difficult  to  learn,  or  too  confusable  to  be  useful  in  the  field.  Symbol 
preference  was  the  third  criterion  used  to  select  between  highly  similar  symbols.  This  preference 
variable  represented  the  participants’  preference  between  the  two  choices  for  a  facility  or  service. 
The  number  in  the  user  preference  column  indicates  the  number  of  participants  who  preferred  the 
symbol.  Totals  of  less  than  28  occurred  for  some  symbols  because  some  participants  did  not 
express  a  preference. 


'  The  researchers  initially  considered  recall  data  for  analysis  purposes  until  discovering  that  long,  difficult,  or 
unfamiliar  names  resulted  in  errors  that  had  little  to  do  with  how  well  the  symbol  represented  the  service  or  facility. 
As  an  example,  the  Weather  Message  Switching  Center  Replacement  (WMSCR)  symbol  had  three  and  four  recall 
errors  respectively  but  no  recognition  errors.  Therefore,  they  eliminated  recall  errors  from  the  final  analysis. 
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Table  2.  Symbol  Selection 


Group  A 


Facility/Service 


Symbol 


Recognition 
Errors 


Average 

Rating 


User 
Preferen 


Automated  Flight 

Service  Station 
(AFSS) 


0 


5.57 

(SD=1.59) 


12 


Air  Ground 
Communications 
Facilities  (AGCF) 


U 


5.90 

(SD=1.28) 


Airport 


6.21 

(SD=1.36) 


N/A 


Automated  Radar 
Terminal  System 
(ARTS) 


A 


4.21 

(SD=1.85) 


13 


Airport  Surveillance 
Radar  (ASR) 


4.86 

(SD=1.84) 


16 


Air  Traffic  Control 
Beacon  Interrogator 
(ATCBI) 


4.14 

(SD=2.04) 


Automated  Weather 
Observation  System 
(AWOS)/ 
Automated  Surface 
Observing  System 
(ASPS) _ 


Tp  ^ 
fol  : 

PI 


5.35 

(SD=1.53) 


12 


Central  Computer 
Complex  Host 
(CCCH) 


&a 


4.76 

(SD=1.53) 


Computer  Display 
Channel  (CDC)/ 
Display  Channel 
Complex  (DCC) 


ma 


4.48 

(SD=1.65) 


12 


Direct  Access  Radar 
Channel  (DARC) 


□a 


5.07 

(SD=1.68) 


13 


Direction  Finder 
(DF) 


6.33 

(SD=0.84) 


15 


Distance  Measuring 
Equipment  (DME) 


f 


5.95 

(SD=0.89) 


19* 


Environmental 
Systems  (EVS) 


6.43 
(SD=1.16) 


19 


1 

Group  B 

ice 

Symbol 

Recognition 

Errors 

Average 

Rating 

User 

Preference 

0 

5.74 

(SD=1.42) 

16 

Irzal 

0 

5.90 

(SD=1.25) 

21* 

DQ 

0 

3 

5.21 

(SD=2.01) 

14 

1 

5.36 

(SD=1.70) 

12 

d3M 

1 

4.50 

(SD=1.42) 

20* 

1 

1 

4.98 

(SD=1.69) 

16 

i 

■111 

i 

0 

5.26 

(SD=2.00) 

19* 

EO 

3 

4.52 

(SD=1.91) 

13 

2 

4.59 

(SD=1.70) 

13 

0 

5.86 

(SD=1.19) 

11 

1 

1 

1 

5.45 

(SD=1.48) 

8 

1 

6.45 

(SD=0.64) 

8 

7 


Group  A 


Group  B 


Precision  Approach 
Radar  (PAR) 


Precision  Runway 
Monitor  (PRM) 


Region 


Tactical  Air 
Navigation  at  VOR 
(TACR) 


Terminal  Automated 
Radar  Service 
(TARS) 


Terminal  Radar 
Approach  Control 
(TRACON) 


Terminal  Radar 
Service  (TRAD) 


QOfl 


Facility/Service  Symbol  Recognition 

Errors 

_ _ 


LORAN  C  Monitor  1 

(LRNCM)  i  r  0 


Markers 


Microwave 
Communications 
Systems  (MCS) 


Maintenance 
Processing  System 
(MPS) 


National  Air  Space 
Data  Interchange 
Network  (NADIN) 


Non-Directional 
Beacon  (NDB) 
Locator  Outer 
Marker  (LOM) 


Average 

Rating 

User 

Preference 

5.02* 

(SD=0.83) 

14 

1j 

6.21 

(SD=0.99) 

18* 

6.50* 

(SD=:0.62) 

11 

4.74 

(SD=1.60) 

14 

6.33* 

(SD=0.70) 

11 

5.67 

(SD=1.42) 

19* 

5.83 

(SD=1.03) 

10 

6.40 

(SD=0.82) 

17 

. 

5.04 

(SD=0.87) 

19* 

4.64 

(SD=1.49) 

11 

14 

4.00 

(SD=1.72) 

5 

16 

4.83 

(SD=1.88) 

5 

5 

4.90 

(SD=1.89) 

6 

User 

Preference 
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Group  A 

Group  B 

Facility/Service 

Symbol 

Recognition 

Errors 

Average 

Rating 

User 

Preference 

Symbol 

Recognition 

Errors 

Average 

Rating 

User 

Preference 

Telecommunications 
Systems  (TS) 

A 

2 

6.07* 

(SD=1.33) 

17 

k  d 

4 

5.00 

(SD=1.47) 

9 

Visual  Aids  (VA) 

0 

6.52* 

(SD=0.87) 

13 

H 

0 

5.59 

(SD=1.41) 

13 

Voice  Systems  (VS) 

0 

5.88 

(SD=1.36) 

13 

0 

5.33 

(SD=1.48) 

14 

Weather  Message 
Switching  Center 
Replacement 
(WMSCR) 

(CO 

jioio  jiOlO 

0 

4.52 

(SD=1.57) 

12 

o 

0 

4.69 

(SD=1.52) 

14 

The  Recognition  Errors  column  in  Table  2  shows  the  total  number  of  recognition  errors  summed 
over  all  participants  in  that  group.  The  Average  Rating  column  shows  the  means  and  standard 
deviations  of  the  rating  data  over  all  the  participants  in  a  group.  Asterisks  denote  statistical 
significance  for  Average  Rating  and  Recognition  Errors  at  p  <  .  10  (student’s  t  test).  For  user 
preference,  asterisks  denote  statistical  significance  for  that  symbol  at  p  <  .05  (Fisher’s  Exact 
Probability  test). 

There  was  no  predefined  maximum  number  of  errors  above  which  a  symbol  would  have  been 
deemed  unacceptable.  Therefore,  the  researchers  looked  at  the  relative  number  of  errors  between 
the  two  groups  rather  than  the  absolute  number  of  errors  to  determine  which  was  the  better  of  the 
two  symbols. 

All  of  the  symbols  exceeded  the  criteria  for  an  average  rating  of  3  or  better,  implying  that, 
although  some  of  the  symbols  are  preferable  to  others,  any  of  the  developed  symbols  are 
acceptable  for  use.  Where  more  than  one  of  the  criterion  for  a  particular  symbol  was  statistically 
significant,  the  statistically  significant  criteria  were  in  agreement  as  to  which  symbol  was  better. 
For  1 1  of  the  services/facilities,  there  was  no  statistical  significance  between  the  two  alternative 
symbols  for  any  of  the  three  measures.  For  these  cases,  SMEs  were  asked  to  make  the  final 
decision  on  which  symbols  should  be  included  in  the  recommended  symbol  set. 

The  Airport  and  Region  facilities  had  only  one  developed  symbol  each.  Both  of  these  symbols 
had  no  errors  and  subjective  rating  data  rated  them  highly. 

3.3  Participant  Comments  and  Resulting  Symbol  Modifications 

The  Exit  Questionnaire  asked  the  participants  for  suggestions  on  how  to  improve  the  symbols. 
This  section  of  the  report  summarizes  the  main  results  of  these  suggestions. 

In  developing  the  candidate  symbols,  the  SMEs  and  researchers  had  combined  similar  facilities 
into  a  single  symbol  to  reduce  potential  symbol  clutter  for  high  level  operations  such  as  an 
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operational  control  center  (OCC)  display.  For  example,  a  Marker  symbol  reflected  the  Fan 
Marker  (FM),  Inner  Marker  (BM),  Middle  Marker  (MM),  and  Outer  Marker  (OM).  The  test 
participants  frequently  commented  that  the  grouped  symbols  were  more  difficult  to  recall. 
Eventually,  consolidated  groups  such  as  markers  and  visual  aids  may  require  specific  symbols 
representing  specific  equipment  to  access  lower  level  menus. 

The  participants  recommended  standardizing  symbol  components  such  as  tower  shading  and 
antenna  shapes.  Participants  commented  that  symbols  such  as  TARS,  TRAD,  ARTS,  and  ASR 
were  difficult  to  learn  due  to  their  similarity  with  other  symbols.  They  suggested  the  addition  of 
an  “s”  to  the  TARS  and  TRAD  symbols  to  reduce  the  confusion  factor  for  service  related 
symbols  by  making  them  unique  to  the  service  category. 

4.  Conclusion 


The  results  of  the  data  analysis  suggest  that  the  32  visual  symbols  presented  in  Table  3  are 
effective  for  representing  services  and  facilities  on  AF  displays.  System  designers  can  use  these 
symbols  to  standardize  the  AF  environment.  Evaluated  by  potential  users,  they  are  likely  to 
perform  well  in  future  AF  environments. 

This  study  did  not  address  the  issue  of  symbol  size.  Additional  research  may  be  needed  to 
determine  the  optimal  symbol  size.  In  future  operational  displays,  users  may  need  to  adjust 
symbol  sizes  to  accommodate  large  quantities  of  data.  It  will  be  necessary  to  ascertain  that  the 
symbols  are  not  too  small  for  the  viewing  distance.  Future  research  could  specify  the  minimum 
recommended  symbol  size.  In  addition,  future  research  could  determine  the  optimal  size 
necessary  to  minimize  eyestrain  and  to  accommodate  users  with  vision  limitations. 
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Table  3.  Recommended  Symbol  Set 


Acronym 


AFSS 

Automated  Flight  Service 
Station 


AIRPORT 

Airport 


ASR 

Airport  Surveillance  Radar 


(consolidated  facilities) 
AWOS/ASOS 

Automated  Weather  Observing 
System/  Automated  Surface 
Observing  System 


(consolidated  facilities) 
CDC/DCC 

Computer  Display  Channel  / 
Display  Channel  Complex 


DF 

Direction  Finder 


EVS 

Environmental  Systems 


(consolidated  facilities) 
MARKERS 


MPS 

Maintenance  Processing 
System 


Recommended 


m 


Acronym 


(consolidated  facilities) 
AGCF 

Air  Ground  Communications 
Facilities 


ARTS 

Automated  Radar  Terminal 
System 


ATCBI 

Air  Traffic  Control  Beacon 
Interrogator 


CCCH 

Central  Computer  Complex 
Host 


DARC 

Direct  Access  Radar 
Channel 


DME 

Distance  Measuring 
Equipment 


LRNCM 

Long  Range  Navigation  C 
Monitor 


(consolidated  facilities) 
MCS 

Microwave  Communications 
Systems 


NADIN 

National  Airspace  Data 
Interchange  Network 


Recommended 


la! 
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Acronym 


(consolidated  facilities) 
NDB/LOM 

Non  Directional  Beacon  / 
Locator  Outer  Marker 


PAR 

Precision  Approach  Radar 


RDAT 

Radar  Data  Service 


TACR 

Tactical  Air  Navigation  at 
VHP  Omidirectional  Range 
(VOR) 


TRACON 

Terminal  Radar  Approach 
Control 


(consolidated  facilities) 

TS 

Telecommunications  Systems 


(consolidated  facilities) 
VS 

Voice  Systems 


Recommended 


Acronym 


ODAPS 

Oceanic  Display  And 
Planning  System 


PRM 

Precision  Runway  Monitor 


Recommended 


REGION 

Regions 


(modified) 

TARS 

Terminal  Automated  Radar 
Service 


(modified) 

TRAD 

Terminal  Radar  Service 


(consolidated  facilities) 

VA 

Visual  Aids 

WMSCR 

Weather  Message  Switching 
Center  Replacement 

Acronyms 


AF 

Airway  Facilities 

AFSS 

Automated  Right  Service  Station 

AGCF 

Air  Ground  Communications  Facilities 

ARTCC 

Air  Route  Traffic  Control  Center 

ARTS 

Automated  Radar  Terminal  System 

ASOS 

Automated  Surface  Observing  System 

ASR 

Airport  Surveillance  Radar 

ATCBI 

Air  Traffic  Control  Beacon  Interrogator 

AWOS 

Automated  Weather  Observation  System 

CCCH 

Central  Computer  Complex  Host 

CDC 

Computer  Display  Channel 

DARC 

Direct  Access  Radar  Channel 

DCC 

Display  Channel  Complex 

DF 

Direction  Finder 

DME 

Distance  Measuring  Equipment 

EVS 

Environmental  Systems 

FAA 

Federal  Aviation  Administration 

FM 

Fan  Marker 

IM 

Inner  Marker 

LOM 

Locator  Outer  Marker 

LRNCM 

LORAN  C  Monitor 

MCS 

Microwave  Communications  Systems 

MM 

Middle  Marker 

MPS 

Maintenance  Processing  System 

NADIN 

National  Airspace  Data  Interchange  Network 

NDB 

Non-Directional  Beacon 

OCC 

Operations  Control  Center 

ODAPS 

Oceanic  Display  and  Planning  System 

OM 

Outer  Marker 

PAR 

Precision  Approach  Radar 

POCC 

Prototype  Operations  Control  Center 

PRM 

Precision  Runway  Monitor 

RDAT 

Radar  Data  Service 

SME 

Subject  Matter  Expert 

TACR 

Tactical  Air  Navigation  at  VOR 

TARS 

Terminal  Automated  Radar  Service 

TRACON 

Terminal  Radar  Approach  Control 

TRAD 

Terminal  Radar  Service 

TS 

Telecommunications  Systems 

VA 

Visual  Aids 

VOR 

Very  High  Frequency  Omnidirectional  Range 

VS 

Voice  Systems 

WMSCR 

Weather  Message  Switching  Center  Replacement 
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Appendix  A 

Combined  Facility  Symbols 


Individual  Facilities 

Combined  Facility 

Backup  Emergency  Communications  (BUEC),  Remote  Communications 

Air/Ground 

Air/Ground  (RCAG),  Remote  Communications  Outlet  (RCO),  and  Remote 

Communications 

Transmitter/Receiver  (RTR) 

Facilities  (AGCF) 

Air  Traffic  Control  Radar  Beacon  (ATCRB),  and  Mode-S 

ATCRB/MODES 

Automated  Weather  Observing  System  (AWOS)  and  Automated  Surface 
Observing  System  (ASOS) 

AWOS/ASOS 

Computer  Display  Channel  (CDC)  and  Display  Channel  Complex  (DCC) 

CDC/DCC 

Central  Control  Monitoring  Systems  (CCMS)  and  Environmental  Remote 

Environmental  Systems 

Maintenance  Systems  (ERMS) 

(EVS) 

Fan  Marker  (FM),  Inner  Marker  (IM),  Middle  Marker  (MM),  and  Outer 
Marker  (OM) 

Markers  (MARKER) 

Radio  Communications  Link  Repeater  (RCLR),  Radio  Communications 

Microwave 

Link  Terminal  (RCLT),  Radar  Microwave  Link  Repeater  (RMLT),  Radar 

Communications  Systems 

Microwave  Link  Terminal  (RMLT),  Television  Microwave  Link  Repeater 
(TMLT),  Television  Microwave  Link  Transmitter  (TMLT),  and  Television 
Microwave  Link  Indicator  (TMLI)  | 

(MCS) 

Non>Directional  Beacon  (NDB)  and  Locator  Outer  Marker  (LOM) 

NDB/LOM 

Data  Multiplexing  Network  (DMN)  and  Leased  Interfacility  NAS 

Telecommunications 

Communications  Systems  (LINCS) 

Systems  (TS) 

Approach  Light  System  (ALS),  Medium  Intensity  ALS  (MALS),  Precision 
Approach  Path  Indicator  (PAPI),  Runway  End  Identification  Lights  (REIL), 
Visual  Approach  Slope  Indicator  (VASI),  and  MALS  with  Runway 
Alignment  Indicator  Light  (MALSR) 

Visual  Aids  (VA) 

Integrated  Communications  Switching  System  (ICSS)  and  Voice  Switching 
and  Control  System  (VSCS) 

Voice  Systems  (VS) 
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Appendix  B 

Subjective  Data  Collection  Sample  Screen 


Subjective  Questionnaire  -  [Evaluation  ol  Symbols] 


Precision  Rumvay  Monitor 


[ow  well  do  you  think  that  this  symbol  represents 
le  Facility/Service  it  stands  for  ? 


Eow  dfficult  do  you  think  that  this  Symbol  will  be 
for  an  AF  personnel  to  learn  ? 


Appendix  C 
Exit  Questionnaire 


Name: 

User  ID: 

Date: 

1 .  Did  you  find  the  entire  testing  process  to  be  mentally  demanding? 


2.  Do  you  think  there  were  too  many  symbols  to  leam  or  could  you  have  learned  a  few  more? 


3.  What  do  you  think  of  the  methods  of  training?  Do  you  have  any  suggestions  for 
improvement? 
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4.  Is  there  any  particular  symbol  you  dislike?  If  yes,  do  have  any  suggestions  for 
improvement? 


5.  Can  you  circle  the  symbol  you  prefer  for  each  facility  or  service  (out  of  the  two 
alternatives),  on  the  attached  sheets? 


Note:  The  following  symbols  are  organized  by  group.  During  testing  they  were  randomly 
presented.  Changes  and  modifications  were  made  to  some  of  the  symbols  so  the  recommended 
symbol 


Airport 


AFSS 


AGCF 


ARTS 


ASR 


ATCBI 


AWOS 


CCCH 


Group  A  Group  B 
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